No-reference Video Quality Assessment for Noise, Blur, and MPEG2 Natural Videos

In this paper, we propose a new no-reference VQA metric, called Video Hybrid No-reference (VHNR) method. It is based on natural video statistics built from the coefficients of 3D curvelet and cosine transforms. VHNR can blindly predict the quality of noisy, blurry, or MPEG2 compressed videos and requires no original reference video. The 3D curvelet transform is known to be sensitive to surface singularities, generated by noise or blur artifacts in the videos. On the other hand, the cosine transform is well-suited to detect MPEG2 compression artifacts. No-reference works because we studied tens of thousands of distorted videos and obtained a statistical relation between the video quality, the specific video characteristics in the transformed spaces, and the video motion speed. Intensive computations are required to analyze tens of thousands of simulated high resolution videos. Since 3D curvelet transform of each video requires 6GB memory and large amounts of computation time, the algorithm is implemented on the FSU High-Performance Computing (HPC) using MPI. The parallelism reduces the computational time of the whole experiment from 118 days to 9 days (a speedup of 13). Introduction of Video Quality Assessment When taking videos by a digital camcorder, there are always video distortions, for example, noise, blur, compression, or some combination thereof. How does one quantify such distortions, or the video quality? A video quality assessment (VQA) metric analyzes the video and assigns a numerical score to the video quality. There are three categories of VQA metrics: full-reference, reduced-reference, and no-reference. Full-reference methods, such as PSNR, compare the full set or subset of original high quality video and the distorted video. However, the original video often does not exist, such as YouTube videos or video taken by low quality camcorders. In such cases, no-reference VQA is the only method to give the quality score. Most no-reference VQA analyze the artifact patterns in the video; therefore the methods usually only work for one particular type of filter. The proposed algorithm has been applied successfully to images filtered by noise, blur, or MPEG2. Introduction of 3D Curvelet Transform We transform the videos into curvelet space to retrieve the features for each video. The curvelet transform is a recent transform which has been proved to have the best convergence properties when representing the surface-like singularities. In a 3D Cartesian grid, the 3D curvelet transform [1, 2] is defined as where and . We can think of the 3D curvelet transform as a convolution of the curvelet function and the video function. A curvelet decays along two axes and is oscillatory along the third orthogonal axis. And it has multi-scale, multi-angle, and multi-location. If a curvelet overlaps a surface singularity whose shape, angle and location correlates well with the curvelet, the corresponding curvelet coefficient is large; otherwise, the coefficient is nearly zero. This property is used to advantage when analyzing filtered videos.